CLAIMS 

We claim: 

1. In a computer system that includes software under test, a method of 
verifying the software with one or more tunable test cases that are capable of being set 
to any of a plurality of verification levels, the method comprising acts of: 

reading in one or more test cases that include a plurality of software testing 
instructions organized as a plurality of verification levels within a verification 
hierarchy, wherein at least two verification levels within the verification hierarchy 
define different amounts of checking to perform for determining if the software 
functions as intended when executed; 

reading in verification settings that define one or more desired verification levels 
within the verification hierarchy; 

identifying a portion of the plurality of software testing instructions within the 
one or more test cases that corresponds to the one or more desired verification levels; 
and 

running the portion of the one or more test cases that corresponds to the one or 
more desired verification levels. 

2. The method of claim 1 , wherein a first test case from the one or more test 
cases is part of a first test group, the first test group including one or more software 
testing instructions organized as one or more verification levels within the verification 
hierarchy, and wherein the verification settings that define the one or more desired 
verification levels for the one or more test cases also define one or more desired 
verification levels for the first test group. 
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3. The method of claim 2, further comprising acts of: 

identifying a portion of the one or more software testing instructions within the 
first test group that corresponds to the one or more desired verification levels; and 

running a portion of the first test group that corresponds to the one or more 
desired verification levels. 

4. The method of claim 3, wherein the verification settings define a single 
desired verification level for the first test case and the first test group. 

5. The method of claim 3, wherein the verification settings define a desired 
verification level for the first test case that is different from a desired verification level 
for the first test group. 

6. The method of claim 4, wherein a second test case from the one or more 
test cases is part of the first test group, and wherein the verification settings define a 
desired verification level for the second test case different from the desired verification 
level for the first test group. 

7. The method of claim 5, wherein a second test case from the one or more 
test cases is part of the first test group, and wherein the verification settings define a 
desired verification level for the second test case that is different from the desired 
verification level for the first test group. 
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8. The method of claim 7, wherein verification settings define a desired 
verification level for the second test case that is different from the desired verification 
level for the first test case. 

9. The method of claim 3, wherein a second test case from the one or more 
test cases is part of the first test group, and wherein third and fourth test cases from the 
one or more test cases are part of a second test group, the second test group including 
one or more software testing instructions organized as one or more verification levels 
within the verification hierarchy, and wherein the verification settings that define the 
one or more desired verification levels for the one or more test cases also define one or 
more desired verification levels for the second test group, the method further 
comprising acts of: 

identifying a portion of the one or more software testing instructions within the 
second test group that corresponds to the one or more desired verification levels; and 

running a portion of the second test group that corresponds to the one or more 
desired verification levels. 

10. The method of claim 9, wherein the verification settings define different 
desired verification levels for each of the first test case, the second test case, the third 
test case, the fourth test case, the first test group and the second test group. 
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1 1 . The method of claim 10, wherein the first and second test groups are part 
of a third test group, the third test group including one or more software testing 
instructions organized as one or more verification levels within the verification 
hierarchy, and wherein the verification settings that define the one or more desired 
verification levels for the one or more test cases also define one or more desired 
verification levels for the third test group, the method further comprising acts of: 

identifying a portion of the one or more software testing instructions within the 
third test group that corresponds to the one or more desired verification levels; and 

running a portion of the third test group that corresponds to the one or more 
desired verification levels. 

12. The method of claim 11, wherein the verification settings define a 
desired verification level for the third test group different from each of the first test 
case, the second test case, the third test case, the fourth test case, the first test group and 
the second test group. 



13. The method of claim 1, wherein at least a portion of at least one of the 
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14. The method of claim 13, wherein the software information available is 
debug information. 
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15. The method of claim 1, wherein the portion of the one or more test cases 
that corresponds to the one or more desired verification levels does not produce any 
testing output. 

16. The method of claim 1 , wherein the portion of the one or more test cases 
that corresponds to the one or more desired verification levels produces one or more test 
outputs for verifying the software. 
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17. In a computer system that includes software under test, a method of 
verifying the software with one or more tunable test cases that are capable of being set 
to any of a plurality of verification levels, the method comprising steps for: 

loading one or more test cases that include a plurality of software testing 
instructions organized as a plurality of verification levels within a verification 
hierarchy, wherein at least two verification levels within the verification hierarchy 
define different amounts of testing to perform for determining if the software functions 
as intended when executed; 

receiving verification setting instructions for one or more desired verification 
levels from within the verification hierarchy for use in testing the software; and 

testing the software at the one or more desired verification levels by running the 
one or more test cases that include the plurality of software testing instructions that 
correspond to the one or more desired verification levels. 

18. The method of claim 17, wherein a first test case from the one or more 
test cases is part of a first test group, the first test group including one or more software 
testing instructions organized as one or more verification levels within the verification 
hierarchy, and wherein the verification setting instructions for the one or more desired 
verification levels define a desired verification level for the first test case different from 
a desired verification level for the first test group, further comprising acts of: 

identifying a portion of the one or more software testing instructions within the 
first test group that corresponds to the one or more desired verification levels; and 

running a portion of the first test group that corresponds to the one or more 
desired verification levels. 
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19. The method of claim 18, wherein a second test case from the one or 
more test cases is part of the first test group, and wherein the verification setting 
instructions define a desired verification level for the second test case that is different 
from a desired verification level for the first test group. 

20. The method of claim 19, wherein verification setting instructions for the 
desired verification levels define a single verification level for the first and second test 
cases. 
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21. In a computer system that includes software under test, a computer 
program product comprising one or more computer readable media carrying computer 
executable instructions that implement a method of verifying the software with one or 
more tunable test cases that are capable of being set to any of a plurality of verification 
levels, the method comprising acts of: 

reading in one or more test cases that includes a plurality of software testing 
instructions organized as a plurality of verification levels within a verification 
hierarchy, wherein at least two verification levels within the verification hierarchy 
define different amounts of testing to perform for determining if the software functions 
as intended when executed; 

reading in verification settings that define a desired verification level within the 
verification hierarchy; 

identifying a portion of the plurality of software testing instructions within the 
one or more test cases that corresponds to the desired verification level; and 

running the portion of the one or more test cases that corresponds to the desired 
verification level. 
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23. The computer program product of claim 22, wherein the verification 
settings define a single desired verification level for the first test case and the first test 
group, and wherein a second test case from the one or more test cases is part of the first 
test group, and wherein the verification settings define a desired verification level for 
the second test case different from a desired verification level for the first test group. 

24. The computer program product of claim 22, wherein a second test case 
from the one or more test cases is part of the first test group, and wherein third and 
fourth test cases from the one or more test cases are part of a second test group, the 
second test group including one or more software testing instructions organized as one 
or more verification levels within the verification hierarchy, and wherein the 
verification settings that define the one or more desired verification levels for the one or 
more test cases also define one or more desired verification levels for the second test 
group, and wherein the verification settings define different desired verification levels 
for each of the first test case, the second test case, the third test case, the fourth test case, 
the first test group and the second test group. 
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25. In a computer system that includes software under test, a computer 
program product comprising one or more computer readable media carrying computer 
executable instructions that implement a method of verifying the software with one or 
more tunable test cases that are capable of being set to any of a plurality of verification 
levels, the method comprising steps for: 

loading one or more test cases that include a plurality of software testing 
instructions organized as a plurality of verification levels within a verification 
hierarchy, wherein at least two verification levels within the verification hierarchy 
define different amounts of checking to perform for determining if the software 
functions as intended when executed; 

receiving verification setting instructions for one or more desired verification 
levels from within the verification hierarchy for use in testing the software; and 

testing the software at the one or more desired verification levels by running the 
one or more test cases that include the plurality of software testing instructions that 
correspond to the one or more desired verification levels. 

26. The computer program product of claim 25, wherein a first test case 
from the one or more test cases is part of a first test group, the first test group including 
one or more software testing instructions organized as one or more verification levels 
within the verification hierarchy, and wherein the verification setting instructions for the 
one or more desired verification levels define one or more desired verification levels for 
both the first test case and the first test group. 
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27. The computer program product of claim 26, further comprising acts of: 
identifying a portion of the one or more software testing instructions within the 

first test group that corresponds to the one or more desired verification levels; and 

running a portion of the first test group that corresponds to the one or more 
desired verification levels. 

28. The computer program product of claim 27, wherein the verification 
setting instructions define a single desired verification level for the first test case and the 
first test group. 

29. The computer program product of claim 27, wherein the verification 
setting instructions define a desired verification level for the first test case that is 
different from a desired verification level for the first test group. 

30. The computer program product of claim 28, wherein a second test case 
from the one or more test cases is part of the first test group, and wherein the 
verification setting instructions define a desired verification level for the second test 
case that is different from the desired verification level for the first test group. 

3 1 . The computer program product of claim 29, wherein a second test case 
from the one or more test cases is part of the first test group, and wherein the 
verification setting instructions define a desired verification level for the second test 
case that is different from the desired verification level for the first test group. 
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32. The computer program product of claim 31, wherein verification setting 
instructions define a desired verification level for the second test case that is different 
from the desired verification level for the first test case. 



33. The computer program product of claim 27, wherein a second test case 
from the one or more test cases is part of the first test group, and wherein third and 
fourth test cases from the one or more test cases are part of a second test group, the 
second test group including one or more software testing instructions organized as one 
or more verification levels within the verification hierarchy, and wherein the 
verification setting instructions that define the one or more desired verification levels 
for the one or more test cases also define one or more desired verification levels for the 
second test group, the method further comprising acts of: 

identifying a portion of the one or more software testing instructions within the 
second test group that corresponds to the one or more desired verification levels; and 

running a portion of the second test group that corresponds to the one or more 
desired verification levels. 
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35. The computer program product of claim 34, wherein the first and second 
test groups are part of a third test group, the third test group including one or more 
software testing instructions organized as one or more verification levels within the 
verification hierarchy, and wherein the verification setting instructions that define the 
one or more desired verification levels for the one or more test cases also define one or 
more desired verification levels for the third test group, the method further comprising 
acts of: 

identifying a portion of the one or more software testing instructions within the 
third test group that corresponds to the one or more desired verification levels; and 

running a portion of the third test group that corresponds to the one or more 
desired verification levels. 
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36. The computer program product of claim 35, wherein the verification 
setting instructions define a desired verification level for the third test group that is 
different from the each of the first test case, the second test case, the third test case, the 
fourth test case, the first test group and the second test group. 

37. The computer program product of claim 25, wherein at least a portion of 
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38. The method of claim 37, wherein the software information available is 
debug information. 
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39. The computer program product of claim 25, wherein the portion of the 
plurality of software testing instructions that corresponds to the one or more desired 
verification levels does not produce any testing output. 

40. The computer program product of claim 25, wherein the portion of the 
plurality of software testing instructions that corresponds to the one or more desired 
verification levels produces one or more test outputs used to verify the software 
functions as intended. 
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